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ACCOUNTABLE TALK IN READING 
COMPREHENSION INSTRUCTION 

Mikyung Kim Wolf, Amy C. Crosson, and Lauren B. Resnick 
Learning and Research Development Center, University of Pittsburgh 

Abstract 

This study examined the relationship between the quality of classroom talk and 
academic rigor in reading comprehension lessons. In addition, the study aimed to 
characterize effective questions to support rigorous reading comprehension lessons. The 
data were collected as a part of the Instructional Quality Assessment (IQA) pilot. The 
IQA is a tool-kit being developed to evaluate the quality of instruction. As for the 
indicators of instructional quality, IQA included a set of rubrics to measure the extent to 
which the classroom talk is accountable (Accountable Talk rubrics), the lesson is rigorous 
(Academic Rigor rubrics), and the teacher's expectations are communicated to the 
students (Clear Expectations rubrics). Specifically, Accountable Talk rubrics consist of 
seven dimensions in classroom talk: (1) participation, (2) teacher's linking ideas, (3) students' 
linking ideas, (4) asking for knowledge, (5) providing knowledge, (6) asking for rigorous thinking, 

(7) providing rigorous thinking. The data for this study included 21 reading 
comprehension lessons in several elementary and middle schools from three urban 
school districts. Quantitative analyses showed that the ratings on students' providing 
knowledge and providing thinking rubrics had strong, positive relationships with the rating 
of academic rigor. These results suggest that students' participation in classroom talk 
allows for a rigorous lesson. Qualitatively, the lesson transcripts were closely examined 
to find characteristics of teachers' questions that engage students in high-level thinking. 

This study also discussed implications for effective questioning in classroom and 
effective indicators for instructional quality. 

Classroom talk has drawn an increasing interest among educators and 
researchers as it has come to be viewed as a rich lens for analyzing the role of 
student- teacher interaction in teaching and learning. Not only is talk a primary 
medium of instruction, but it plays a key role in socializing intelligence (Edwards & 
Westgate, 1994; Resnick & Nelson-LeGall, 1999). In light of this, many researchers 
have attempted to characterize patterns of classroom talk (Cazden, 2001; Edwards & 
Westgate, 1994; Heyman, 1983; Lemke, 1990; O'Connor & Michaels, 1993; 1996; 
Tharp & Gallimore; 1991). 

Given the assumption that social interaction supports and increases cognitive 
development in students' learning, it follows that discourse-based instructional 
approaches facilitate effective reading comprehension instruction. These 
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instructional approaches acknowledge that reading comprehension is a collaborative 
process where the teacher and students co-construct meaning from the text 
(Palincsar, 2003). This process-oriented rather than product-oriented approach is 
reflected in collaborative approaches to text comprehension during a discussion 
such as Reciprocal Teaching (Palincsar & Brown, 1984), Questioning the Author 
(Beck, McKeown, Hamilton, & Kucan, 1997), and Collaborative Reasoning (Chinn & 
Anderson, 1998). The common feature of these approaches is that teacher and 
students engage in a dialogue where the teacher models an interaction, focusing on 
supporting students to deepen their understanding of text. Ultimately, the students 
take responsibility to lead the dialogue as they develop into increasingly 
independent readers. 

In traditional lessons, the discussion is typically characterized with a sequence 
of teacher initiation, student response, and teacher evaluation (IRE). The criticism of 
the IRE is that the nature of the question is one that the teacher already knows the 
answer to, so is simply testing the students' knowledge (Cazden 2001). Hence, it 
constrains the participants of the discussion from collaboratively building meaning 
from the text. In contrast, discourse-based lessons focus on the quality of the 
teacher's questions and talk moves to generate effective reading comprehension 
lessons. 

The importance of discussion for comprehending the text has been emphasized 
in a number of studies (Applebee, Langer, Nystrand, & Gamoran, 2003; Beck et al., 
1997; Brown et al., 1993; Goldenberg, 1992; Hiebert &Weame, 1993; Rosenshine & 
Meister, 1994). Beck et al. (1997) have developed a series of queries that the teacher 
can use in assisting students with understanding text in their instructional approach 
called Questioning the Author. The queries were specifically designed to investigate 
the text, grapple with ideas, and clarify ideas in the text. For example, the teacher 
can initiate a discussion by using an open-ended query, "What's going on here?," 
and extends the discussion by asking a follow-up query, "That is what the author 
said, but what did the author mean?" Beck et al. claim that these sets of queries 
reinforce students' comprehension of the text rigorously. These queries also aim to 
teach students to actively self-monitor meaning, including "interrogating" the text, 
and to assist students in making text-based inferences and connecting important 
ideas within the text to build a coherent version of the text. 

Brown et al.'s (1993) study reported that the use of collaborative dialogue 
during the reading comprehension lessons was positively associated with the 
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students' gains in building knowledge and comprehension. On the basis of 
framework of Reciprocal Teaching, this study suggests that the use of dialogic 
strategies such as question generating, summarizing, clarifying, and predicting 
facilitates students' knowledge building and comprehension of the text. Applebee et 
al. (2003) also provided empirical evidence that discussion-based activities coupled 
with academically challenging tasks are positively related to develop students' 
literacy skills. In this study, the researchers specifically focused on the role of 
dialogic instruction where the teacher uses authentic and open-ended questions 
rather than questions for which the teacher has pre-specified answers. 

The effect of specific types of classroom talk has also been studied in some 
research. Hiebert and Wearne (1993) examined the particular types of discourse in 
relation to students' responses in length. They found that classrooms with ample use 
of describing and explaining question types (e.g., "Tell how you solved the 
problem." "Explain why a procedure is chosen or why it works.") had a significant 
effect on the gains of academic growth over a school year. Heymen (1983) closely 
examined a specific teacher's utterance, "What do you mean?" and the students' 
diverse responses to it. Based on a thorough discourse analysis, he suggested that 
teachers should make explicit use of this utterance to provide a common stock of 
knowledge for all participants in the classroom. Similarly, O'Connor and Michaels 
(1993, 1996) focused on the value of a particular teacher's strategy, "revoking." 
They argued that the teacher's revoicing efforts provided an opportunity for 
students, themselves to share reformulations. These reformulations became the 
basis on which students then built arguments in classroom discussion. The revoicing 
strategy thus provided a framework for a social interaction to promote classroom 
learning. 

As discussed, there is considerable theoretical support on the links between 
classroom discourse and learning as well as some research on the effects of specific 
conversational strategies. However, empirical research concerned with analyzing the 
coordination of a variety of classroom talk moves and their effects on the lesson is 
scant. 

The present study investigated the relationship between the nature of the 
classroom talk and the degree of the rigor of the lesson task. In particular, this study 
examined both the quality of the teacher's talk and students' talk, and their 
relationship between the two. Further, this study aimed to closely look at the 
teacher-student interaction to distill the characteristics of the teacher's talk moves 
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that facilitate a rigorous discussion which reinforces students' understanding of a 
challenging text or concept and critical thinking. 



Method 

Sample 

Data for this study were collected at two different times. The research team 
visited a number of elementary and middle schools in three urban school districts to 
recruit teachers to volunteer for participation in the study. For the first data 
collection in Spring 2002, we recruited seven teachers in one district, and at the 
second time in Spring 2003, 14 teachers in two school districts agreed to participate. 
In total, 21 teachers from ten schools agreed to allow the research team to observe 
their reading comprehension lessons. Participating teachers were offered a 
bookstore gift certificate as compensation for their time. The range of grade levels 
was from Grade 1 to Grade 8. Parental consent from all students in participating 
teachers' classes was also collected. The total number of students for this study was 
441 of which 45% were male and 55% were female. The participating students were 
diverse in ethnicity, representing 27% African-American, 7% Asian, 48% Latino, 13% 
White, and 5% other. Teachers identified 20% of these students as English language 
learners. 

Instruments 

The data were collected using the Instructional Quality Assessment (IQ A) tool 
developed at Learning and Research Development Center (LRDC) of the University 
of Pittsburgh. The IQA tool was designed to assess the quality of classroom 
instruction in K-12 settings, focusing on four major aspects to promote students' 
learning: (1) accountable talk in the classroom, (2) academic rigor of the lesson, (3) 
clear expectations, and (4) the students' self-management of learning. These aspects 
are empirically and theoretically driven from a great body of research in cognitive 
and social psychology that have been translated by Resnick and her colleagues into a 
set of principles of effective teaching and learning called the Principles of Learning 
(Resnick, Junker, Magruder, Mconachie, & Crosson, 2001; Resnick & Hall, 1998). The 
IQA toolkit consists of 16 rubrics for rating observed lessons, as well as six rubrics 
for rating the quality of assignments that teacher gives to the students in the class. 
All the rubrics except one (level of the text ) are on a 4-point scale (with l=poor and 
4=excellent). The rubric which measures the rigor of the text level has a 3-point scale 
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(with l=poor and 3=excellent). The tool also included student and teacher interview 
protocols and an intensive rater training program. 

This study only dealt with the data from Accountable Talk and Academic Rigor 
rubrics because the purpose was to examine the relationship between classroom talk 
and the rigor of the lesson. A set of Accountable Talk rubrics of the IQA assesses the 
teacher and students' skills for participating in whole group discussions to promote 
learning. Specifically, the Accountable Talk rubrics consist of seven dimensions of 
classroom talk: (1) participation rate, ( 2 ) teacher's linking ideas, (3) students' linking ideas, 
(4) asking for knowledge, (5) providing knozvledge, (6) asking for rigorous thinking, (7) 
providing rigorous thinking. The definitions of these subscales are explained in coding 
of classroom talk section below. Among these, the ratings on 'participation rate' were 
not used for the analysis because this rubric intends to measure only the quantity of 
the classroom talk. With respect to assessing the academic rigor of the lesson, three 
dimensions were rated; (1) rigor of the text, (2) active use of knozvledge: analyzing and 
interpreting the text during the whole-group discussion, and (3) active use of knozvledge 
during the small group or individual tasks. Since the ratings on classroom talk were 
based upon the whole-group discussion, only the ratings on Academic Rigor during 
the whole-group discussion (i.e., the second rubric of Academic Rigor) were used as 
data for this study. 

Classroom Observation 

Two trained raters visited one reading comprehension lesson for each teacher 
participating in the study. The lesson time was about 45-50 minutes. The teachers 
were told to demonstrate a reading comprehension lesson that was as typical as 
possible while including the following components: (1) a text was read aloud to, 
with, or by the students; (2) the teacher lead a whole-group discussion for about 20 
minutes; and (3) the teacher assigned group-work or individual tasks for small- 
group or independent work. By specifying the structure of the lesson for each 
teacher, we attempted to reduce the variability across the lessons due to differences 
in diverse lesson structures. In our conversations with teachers before and after 
lesson observations, most teachers indicated that the observed lesson was typical of 
their daily practice. Those who commented that they had to alter their lesson to 
accommodate the IQA research team said that most of their reading comprehension 
work was with smaller groups of students, but that the lesson observed — although 
with a larger group than normal — still represented typical teaching. Teachers 
completed a pre-visit questionnaire, including information about the observed 
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lesson in advance of the observation in order to provide background information for 
the raters. 

The raters rated the quality of classroom talk based only on the whole-group 
discussion. This is because it was not feasible for raters to capture the teacher and 
students' interaction during the small group or individual activities. However, the 
academic rigor of the lesson was rated holistically based on the entire lesson 
observation. 

All the lessons were audiotaped and transcribed. Additionally, the raters made 
detailed notes during their observation, which they used as evidence to justify their 
scores. They transcribed the class interaction during the whole group discussion. 
The raters assigned a score for each rubric right after the lesson observation, and the 
raters then debriefed their scores. When disagreement between scores occurred, 
raters compared evidence to reach a consensus score. 

Coding of Classroom Talk 

Accountable Talk rubrics have been developed with consideration of three 
dimensions that are important for learning: (1) accountability to learning community 
(ALC), (2) accountability to accurate knowledge (AAK), and (3) accountability to 
rigorous thinking (ART). 

The purpose of ALC is to measure the degree to which teachers and students 
make consistent efforts to ensure that all participants understand the ideas and 
positions shared during the whole-group discussion. Additionally, this dimension 
measures the extent to which teachers and students make efforts to link 
contributions to one another so that the discussion builds on ideas within the 
learning community. To receive a high score on the rubrics for this dimension, 
whole-group discussion would have to evidence that teachers/ students are listening 
attentively, paraphrasing, and building ideas upon one another's contributions. 

The purpose of AAK is to measure the degree to which teachers and students 
ensure that participants provide specific and accurate knowledge as evidence to 
back up their contributions. In other words, this dimension captures the extent to 
which there is a "commitment to getting the facts right" in the lesson discussion. To 
receive a high score on rubrics for this dimension, whole-group discussion would 
have to evidence that teachers press for accurate knowledge by asking participants 
to support their contributions, and that students, in turn, support their contributions 
with specific, appropriate, and accurate evidence. 
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The purpose of ART is to measure the degree to which speakers are asked to 
explain their thinking by using rational strategies to present arguments and by 
drawing logical conclusions. This dimension also measures the degree to which 
contributors explain their thinking. To receive a high score on rubrics for this 
dimension, classroom talk would have to evidence that students are pressed to 
explain thinking for their claims, and students would have to explicate their 
reasoning in ways appropriate to the discipline. 

Table 1 illustrates specific rubrics under these three dimensions and a few 
examples of talk moves that the raters are trained to code during the lesson 
observation. 

During the rater training program, the raters studied numerous video clips to 
become familiar with accountable talk moves and to be able to identify them 
appropriately. 

Analyses 

Since this paper is primarily concerned with the type of classroom talk moves 
and the level of rigor during the whole group discussion, only the scores on the 
accountable talk moves and the academic rigor of the lesson during the whole group 
discussion were used in the analyses. In order to examine the reliability of the raters' 
scores, the percentage of agreement on the exact scale point was first calculated. In 
addition, Cronbach's alpha coefficients and Spearman's correlation coefficients were 
computed to examine the internal consistency of the ratings and relationship 
between the two raters' scores. 

Consensus scores were used to investigate the relationships between scores on 
the accountable talk and academic rigor rubrics. Correlations analyses were 
conducted in order to investigate the relationships among each rubric. Stepwise 
regression was also performed to discover which rubric had the most predictability 
of scores for the academic rigor rubric. All statistical analyses were conducted using 
SPSS 11.0 (2001). 

As for qualitative analyses, the transcripts and the raters' notes were closely 
examined in order to characterize the patterns of the interaction and the teacher's 
“talk moves.” Specifically, a comparison was made between a weak and a strong 
quality of particular kinds of talk moves made by teacher and students. 
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Table 1 

Category of Accountable Talk 

Category (Rubric) Description Talk moves’ 1 ' 

Accountability to Learning Community (ALC) 

Teacher's linking (T: L) To what extent does the • John, did you hear what 

teacher explicitly link different Ann just said? Can you 

people’s ideas? repeat that in your own 

words? 

• Who agrees / disagrees 
with what Ann just said? 

• Who wants to add on to 
what Ann just said? 

• I want to add on to what 
Ann said.... 

• I agree with you 
because... 

Accountability to Accurate Knowledge (AAK) 

Asking for knowledge (T: K) To what extent does the • How do you know that? 

teacher press the students to 

support their ideas with * Can you give me some 

evidence based on the text? examples? 

• Where did you find that 
information? 

• Can you show me which 
part of the text tells you 
that information? 

• What do you mean? 

To what extent do the • I know that because it 

students support their ideas says here. . . 

with evidence based on the 
text? 



Accountability to Rigorous Thinking (ART) 


Asking for rigorous thinking 


To what extent does the 


• 


Why do you think that? 


(T: Th) 


teacher press the students to 
explain their thinking? 


• 


Can you explain that 
more? 






• 


Say more about that. 


Providing with rigorous 


To what extent do the 


• 


I think because. . . 


thinking (S: Th) 


students explain their 
thinking? 







’‘'Accountable Talk and Talk moves are well-described in the Accountable Talk SM CD (2002) 
produced by the Institute for Learning (IFL) / LRDC. 



Providing with knowledge 
(S: K) 



Student's linking (S: L) To what extent does the 

student explicitly link 
different people’s ideas? 
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Results 



Interrater Reliability 

Regarding accountable talk scores, 64% of exact agreement was obtained 
between the two raters. Cronbach's alpha coefficient was .89, indicating high 
internal consistency of the scores. In addition, the Spearman's correlation coefficient 
was .80 (p < .00 at a = .05), suggesting that there was a moderate level of the rating 
tendency between the two raters' scores. Slightly higher rater reliability was found 
in academic rigor scores. The exact agreement between the two raters was 76%. 
Cronbach's coefficient alpha was .93, and the Spearman's correlation coefficient was 
.88 (p < .00 at a = .05). 

IQ A Scores on Accountable Talk and Academic Rigor 

Table 2 presents the means and standard deviations of the scores of the rubrics 
on academic rigor and accountable talk. According to the scales of the IQA tool, a 
score of 3 generally reflects a higher level of cognitive demands and rigor. The 
means of the scores on the academic rigor during the whole group discussion 
indicates that the observed lessons, on average, are slightly beyond the level of 
comprehending the storyline of the text itself and close to the level of analyzing and 
interpreting the text. 

Means of the scores on the accountable talk rubrics ranged from 2.00 to 3.38. 
Linking talk moves had a lowest mean among the accountable talk rubrics. 
Particularly, students' linking talk moves were rarely observed. On the other hand, 
the teachers' efforts to ask students to elaborate their thinking were the most 
frequently observed talk moves. Interestingly, the means on the teacher's 
accountable talk rubrics were higher than their counterparts of the students' talk. 

Relationships Among Accountable Talk and Academic Rigor Scores 

Table 3 provides the results of the correlation analysis among the scores on 
each rubric. Overall, all the accountable talk moves had a significant relationship 
with the rigor of the lesson. In particular, students’ talk moves on knowledge and 
thinking had the highest correlations with the ratings on the academic rigor (.84 for 
students' knowledge and .80 for student's thinking). It is also notable that the teacher's 
asking for knozvledge and the students' providing knozvledge was highly intercorrelated. 



9 




as was true for the relationship between teacher's asking for rigorous thinking and the 
students' providing rigorous thinking (.90 for knowledge and .88 for rigorous thinking). 



Table 2 

Mean and SD for the Scores on Each Rubric (N = 21 lessons) 



Rubric 




M 




SD 


Academic Rigor 










Active use of knowledge 




2.67 




.966 


Accountable Talk 










Teacher's linking 




2.14 




.910 


Student's linking 




2.00 




1.183 


Asking knowledge 




3.14 




.854 


Providing knowledge 




3.10 




1.091 


Asking rigorous thinking 




3.38 




.921 


Providing rigorous thinking 




3.14 




1.108 


Table 3 










Correlations Among the Scores on the 


Rubrics of AR and AT ( N 


= 21) 




AR T:L S:L 


T:K 


S:K 


T:Th 


S:Th 


AR - .57** .61** 


79** 


.84** 


.60** 


.80** 


T:L - .42 


.42 


.49* 


.17 


.33 


S:L 


.45* 


.47* 


.32 


.50* 


T:K 


- 


.90** 


.56** 


72** 


S:K 




- 


.36 


.65** 


T:Th 






- 


.88** 


S:Th 








- 


* significant at a - .05, ** significant at a = .01. 

In order to investigate which talk move has the highest predictability of the 
ratings on the academic rigor of the lesson, stepwise regression analyses were 
conducted. Although all six accountable talk rubrics were entered as predictors, only 
two rubrics were found to be significant predictors of the scores on the academic 
rigor during the lesson. As shown in Table 4, the students ’ providing knozvledge and 



providing rigorous thinking were the only significant predictors. Particularly, the 
students ’ providing knowledge was the highest predictor of the holistic scores, with a 
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standardized regression weight of .84. The R 2 indicated that 81% of the variance of 
the academic rigor scores was accounted for by the combination of these two talk 
moves. 



Table 4 

Summary of Stepwise Regression Results Predicting AR Scores (N = 21) 





Variable 


R 2 


Adjusted R 2 


R 2 

Change 


Si g 


B 


SEB 


P 


Step 1 


S: K 


.70 


.69 


.70 


.00 


0.74 


.11 


.84* 


Step 2 


S: K 


.81 


.79 


.11 


.00 


0.49 


.12 


.56* 




S: Th 










0.38 


.12 


.43* 



*p < .05 



Comparison of Weak and Strong Accountable Talk Moves 

Qualitatively, we examined the characteristics of the talk moves that the 
raters had coded from their field notes. The following selected examples describe 
“weak” and “strong” quality of talk moves for the three dimensions of accountable 
talk. Italics in the following excerpts indicate the talk moves that overtly 
corresponded to each dimension. 

Accountability to Learning Community: Linking. As seen in Table 2, linking 
talk moves were the least frequently observed. It appears that most linking talk 
moves stayed at the superficial or formulaic level. Examples (1) and (2) display weak 
teacher's linking moves. 

(1) Example of weak linking (Grade 2) 

T: Question Two. Why did he want so badly to change back to himself? 

S: Big Anthony had this wish because he wanted the ladies to stop chasing him. 

T: OK. Do you guys agree zvith him? Yes, he wanted the ladies to stop chasing him and I, I'm 

not sure he's quite happy about this thing anymore. 

(2) Example of weak linking (Grade 5) 

[One student makes prediction.] 

T: Right. Okay, so by the title he's making a prediction. He's thinking that it's about a 

shipwreck. Good prediction. Let's have one more prediction made, because we want to 
get to the story. Louis, what do you think? 

S3: I think it's gonna be [unintelligible] about people that [unintelligible] look for whales. 

T: So, you think it's about people that are gonna look for whales. So how many people 

think — how many people agree zvith Marco or zvhat Louis says ? It's either about people who 
are gonna look for whales, or about a shipwreck. Kay, so do zve agree zvith one of them? 
Those are two totally different predictions, aren't they? 

Let's find out. Let's see what the story's about. And I like Chris Van Allsburg; he's one of 
my favorites. 

[Teacher reads.] 
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In both examples, the teachers used an explicit linking talk moves such as "Do 
you guys agree with him?/' "How many people agree with Marco?," and "Do we 
agree with one of them?" However, the teachers gave little or no wait time for 
students to respond. Instead, the teachers answered their own questions. 
Additionally, the questions themselves were limited to yes-no questions, and did 
not require a response from a specific student. 

On the other hand, in example (3), not only did the teacher provide an 
opportunity for students to build on each other's ideas (“Do you wanna add...?"), 
but also she reformulated what the student had said (“So what I hear you saying 
is...”). Note that once one student (S2) added his own thought to the other's (SI), 
the teacher summarized what the student said by starting her utterance, "So." This 
example of linking illustrates a discussion that supports the development of a 
learning community where students are able to follow each other's ideas. 

(3) Example of strong linking (Grade 4) 

SI: I talked about Larry Brown (unintelligible) I said that I made text to text connection with 

Larry Dunn and Rufus because he felt like he wanted to cry but was tough — Rufus was 
too. And because he [Rufus] doesn't have money to buy sandwiches, Rufus's best friend 
was laughing at him. 

T: So zvlwt I hear you saying is that you're making a character trait connection. Both 

characters have that trait of pride they don't want to show their true feelings that they're 
hurt and I agree with you. Someone on the other side of the room want to share with me? 
Do yon wanna add something new! 

S2: I'd like to add to David because they both have ripped clothes, and pride. Now they're 

both getting beat up. Larry and Rufus are alike. 

T : So they both shared similar experiences. 

A similar example is found in (4). In this excerpt, the students seemed familiar 
with accountable talk moves. In contrast to examples (1) and (2), examples (3) and 
(4) describe cases where the teacher explicitly designated the next speaker. 

(4) Example of strong linking (Grade 5) 

SI: He broke his leg. 

T: He broke his leg. So Victor's actually doing a smart thing. He's inferring information 

from the text. Hozv many others thought that ? Because it said he has a limp. What were yon 
going to say, Derek? 

S2: Umm, I was gonna say that I agree with Victor, because when he was a little boy he wanted 

to sail, um, [unintelligible]. 

T: You know, when he was telling the tale, he said, "The little boy always wanted to sail, 

even in the high winds." Ashley? 

S3: Um, I disagree with Angela and Carl, because when Carl said the boy was gonna fly, 

[unintelligible], and the [unintelligible] started come down. But he hit a tree, and Angela 
said that he, he wasn't, uh, caught in a, um, storm, but he didn't. And she said that he 
wasn't, um, the guy was gonna go [unintelligible]. . . 
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Accountability to Accurate Knowledge (AAK). The raters coded talk moves 
as accountability to accurate knowledge when the teacher and students engaged in 
describing factual information based on the text or supporting one's claim with 
evidence based on the text. The weak AAK examples are as follows in (5)-(7). 

(5) Example of weak AAK (Grade 4) 

[Teacher reads.] 

T: Did he travel the whole world over? 

S: No. He just went around the yard of his house. 

T: And he went to the bam. Is that traveling the whole world over? 

Ss: No. 

T: So we have a problem already right? 

Ss: Yeah. 

In this example, after the teacher read a paragraph, he was checking the 
students' comprehension by asking what exactly happened in the story. Noticeably, 
the students' answers were limited to yes or no, except the first student's response. 
Seemingly, all the teacher's utterances already included the information that the 
students could have inferred from the story. Thus, the teacher provided little room 
for students to make sense of the poem and select appropriate evidence from the text 
to back up their thoughts. 

Example (6) demonstrates a situation where the teacher's utterances contained 
questions that elicited isolated, surface-level facts from the story. As a result, the 
students' utterances merely included one-word responses. 

(6) Example of weak AAK (Grade 6) 

T: I'm gonna go over the workbook while you're working. Number one. What animal was 

at the bottom of the pile? 

SI: The huge dog. 

T: The huge dog. Anybody remember the name of that huge dog? What kind of dog it 

was? 

SI: Greyhound? 

T: No, it wasn't greyhound. What kind of dog was it? Remember I told you guys 

yesterday? 

S2: [inaudible] 

T: No. Saint.... 

S3: Saint Bernard. 

T: Saint Bernard. Good. What dog stood over the huge dog's back legs? 

S4: Spotted dog. 

T: Spotted dog. And S4, how about over his front legs? 

S4: The black dog. 

T: The black dog. How many cats were in the pyramid? 

S5: Four cats? 

T: Four cats. S5, what animals did the cats stand on? 

S5: The dogs? 

T: Right. Well, not just the dogs but the smaller — I took the dogs, but you really should 

have said the smaller dogs. SI, how many squirrels were in the pyramid? 
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SI: Eight? 

T: Eight. 

Another weak example of AAK interaction was found when the teacher 
overscaffoled by providing too much information. In example (7), the teacher's 
second utterance aimed to eliciting the student's ideas on the character, Macduff. 
However, the teacher framed the question in such a way that the student just 
finished the teacher's incomplete sentence. 

(7) Example of weak AAK (Grade 8) 

T: Right, exactly. He's gone to England to get Malcolm. Malcolm is... 

S: Duncan's son. 

T: Duncan's son who ran off to England. So he's going to convince Malcolm and Edward in 

England that they need to raise an army to overthrow Macduff — Macbeth. So therefore, 
in Macbeth's eyes, Macduff is a... 

S: Traitor. 

T: Traitor. And therefore he's not only going to try and go after him, but kill his family. 

OK? So. ..just like in Spiderman. When the Green Goblin says he's going after 
Spiderman's girlfriend. So, the 3 prophecies that the apparitions come up with. One, 
beware Macduff. Two, no man bom of woman can vanquish Macbeth. What does that 
mean? No man born of woman? 

It was found that the teacher's pressing strategy was critical to encourage the 
students' elaboration of their ideas. Example (8) demonstrates that the teacher 
effectively utilized the AAK talk move, "how did you know that?" This question 
extended the student's initial one-word answer (i.e., “No") to a fuller articulation of 
the idea, including citing evidence from the text. In other words, instead of 
confirming the student's response, "The old man is the boy," the teacher provided 
an opportunity for the student to support his idea by repeating the question, "So 
how did you know that?" 

(8) Example of strong AAK (Grade 3) 

T: What are you thinking, Victor? 

S: The old man is the boy. 

T: Hozv did you know that? Did Chris Van Allsburg tell us, in the print, the old man is really 

the boy? 

S: No. 

T: So hozv did yon knozv that, Victor? How did you know that? 

S: Because he said the, umm, that the winds are getting harder, and you have some sailors 

too, and he, he picked up his cane and he started to. . . 

T: So Victor is inferring information from the text. Remember, we talked about that? 

Making inferences? He's taking — Chris Van Allsburg didn't tell us, "This old man is the 
boy." He's using his mind, and you know what? He's inferring information from the 
text, saying, "It says the man had a limp." So, because he has a limp, he's thinking it 
could, it's probably the boy, because what happened to the boy? 

In the following example, the class engaged in an activity of text-to-world 
connections in which students interpreted the story and connected it to a real-life 
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situation. The teacher noted that the student did not provide the part of the story 
that was connected in the student's work. The teacher kept pressing the student to 
provide accurate knowledge from the text (e.g., "Who can retell this part?"). It took a 
number of teacher's attempts to get the students to provide evidence based on their 
knowledge from the text. 

(9) Example of strong AAK (Grade 2) 

S: This part of the story when the house burnt up this book reminds me of when my 

friend's house burnt because they left the light on with a blanket. Now my friend is with 
his Grandmother because his house burnt up. He was sad. He didn't like when his 
house got burnt because you can die he said. 

T: Does that tell you what part of the book? It does? Does it? 

SI: Yes. 

T: Look, this book reminds me of. She did say that. She did tell what happened in the 

world. Did she talk about the part of the book? Did she explain this part? 

Ss: No. 

T: Who can retell that part? 

SI: Tell what? 

T: The part in the book that Amelie, she's making a connection, who can, can you tell that, 

Casey? 

S2: Yeah. This story, this story, this part. . . 

T: What's the part in the story? What's the part in the book, what's the part that she's 

talking about? 

S2: About her, her friend's house burning down. 

T: She's saying the book, the part in the book right? What's that part in the retell? I want 

you to retell me that part. 

S2: Um, the part of the book that reminds me of when my friend's house burnt down. 

T: Did he answer my question? 

Ss: No. 

T: No, what are you supposed to do? Yes, say about the part in the book. When house was 

burnt down. Who can tell the part in the book when the house burnt down? Felix. 

S3: In the book. 

T: Felix, can you take, the book is right there. Can you bring that to me? OK. OK. Does she 

talk about in, in, in her connection? Does she talk about this part? 

S3: No. 

T: Who can retell this part? Who can retell this part of the story? 

S3: When um, when um, the little girl and her mom were walking home they saw the fire 

and they said um, the little girl said, I see fire and I see the um, yeah, smoke and I came 
over to mama and she said how is her grandma and then she said, the author said, that 
author said that it was hard to find the cat. 

Accountability to Rigorous Thinking (ART). Accountability to rigorous 
thinking is concerned with the teacher's efforts to promote students' reasoning to 
support their ideas, and students' elaboration of their logic. Just as in AAK, weak 
ART moves lacked pressing or follow-up questions from the teacher. Examples 
(10a) - (10c) were selected from one second grade lesson. While the teacher read a 
book, she paused occasionally to ask students to explain their thinking. In (10a), 
even though the teacher asked the student to infer the meaning of the phrase, "a 
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long reverie," she never asked how the students figured that out. Similarly, (10b) 
displays the absence of the teacher's follow-up questions or attempts to elicit the 
student's thought. 

(10a) Example of weak ART (Grade 2) 

T: Aha! So you can't see the ball if it is too high and the sun is in your eyes! Okay, what do 

you think this means "a long reverie?" We have never had this word before. 

SI: Dream. 

S2: Daydream. 

S3: Imagining. 

S4: Dream the ball will come to him. 

[Teacher continues to read the book.] 

(10b) Example of weak ART 

T: When I read "They never batted last," what does that make you think? Why does he say 

that? 

S: The boy always bats last. 

[Teacher continues to read the book.] 

In (10c), the teacher assigned a task in which students were asked to interpret 
the underlying meaning of the story. The teacher did not follow up on any student 
contributions when the students presented their ideas. 

(10c) Example of weak ART 

[Teacher reads the book] 

T: Let me read this again so you can discuss this with a partner. [Re-reads stanza and 

students discuss ideas with a partner.] 

T: Okay, let's hear a few ideas. 

SI : I was thinking they hit the ball . . .he is going to get it. 

T: I think I heard something different from this group over here. Would you like to share 

your ideas with us? 

S2: I think he is going to hit a homerun and the team starts to yell because they win the 

game. 

S3: They were yelling because the ball was hit far and they were yelling for him to catch it. 

T: So, you agree with Michele, that they were yelling to tell him to catch the ball. 

S3: Yes, they were yelling at him to pay attention. 

T: Well, let's find out... [Teacher continues to read the book.] 

In contrast, examples (11) and (12) demonstrate the interaction where the 
teacher's utterances included a number of probing questions to press the students to 
explain their thinking. In (11), the teacher scaffolded students to put the main idea in 
their own words. Instead of moving forward after the first student's response, she 
had the students think what the author meant to say about the ocean by asking, "So 
what does that tell us. . .?" and "What can we say about the ocean?" 

(11) Example of strong ART (Grade 3) 

T: I want you to find the main ideas as we're reading. So far, are we correct about the main 

idea? Raise your hand if you can give me a sentence that would tell me the main idea of this 
article. It doesn't have to be a sentence from the article. 
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SI: That over two-thirds of the world's surface is covered by vast ocean. 

T So what does that tell us about the ocean? It's two-thirds. Is two-thirds a little bit of a 
whole? Or most of the whole? 

Ss: Most. 

T: Most of the whole. So what does that tell us about the ocean in general? 

S2: That it's big and gonna take almost half of the world. 

T: So if we say. The ocean is What can we say about the ocean? 

S2: The ocean is big and it takes almost half of the world. 

Example (12) describes the situation where the teacher provoked the 
student's rigorous thinking by repeatedly asking "why" questions. At first, she gave 
an opportunity to the student to explain his idea in more detail by telling him, "tell 
me a little bit more." Upon listening to the student's explanation, the teacher further 
attempted to explore the student's thinking by asking "why the mistreatment," and 
"why?" In consequence, the interaction between the teacher and the student 
accomplished a rigorous and high level of discussion about the student's research. 

(12) Example of strong ART (Grade 7) 

T: Okay, good. So we saw a lot of starvation, we saw assets, things the Jews owned being 

hidden so they wouldn't be taken away. What else? 

SI: How the Nazis treated the Jews. The inhumane activities that went on in the sauna and 

stuff, where they stripped them nude and shaved them, and made them wait in long 
lines in the cold. 

T: Excellent. Inhumanity. Tell me a little bit more about that. 

SI: About inhumanity in general? 

T: Well, in general or in context of the Holocaust or in context of your research. 

SI: My research is on concentration camps. They would import the Jews and, depending on 

what camp it was, they would take them someplace and shave them, strip them naked 
like I said. Then they would put them to work and sometimes if they were caught 
slacking off they would get shot or sent to a torture unit. 

T: So is inhumanity mistreatment? Yes. And we wrote poems about inhumanity, right? 

And how about the Holocaust? Why the mistreatment? 

SI: They were prejudiced against the Jews. They didn't like them. It was just jealousy. 

Anti-semitism. 

T: Why? 

SI: Because anti-semitism is basically jealousy. They didn't like them just because they 

supported a — they had a different kind of religion. 

T: Good. 



Discussion 

This study examined the characteristics of the dialogic interaction in the 
classroom and investigated how specific kinds of talk moves relate to the rigor of the 
reading comprehension lesson. The descriptive statistics show that the means of the 
teacher's talk moves were higher than those of the students' talk moves. This result 
suggests that the teachers, in general, lead the conversation, and used more talk 
moves to obtain student's responses. One interesting finding was that the talk moves 
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regarding knowledge or reasoning were relatively prevalent, but the talk moves of 
participants' linking ideas to each other was not a common practice. According to 
O'Connor and Michaels (1993), revoicing or reformulating each other's ideas is a 
critical means for promoting effective learning. Considering the diverse nature of the 
classroom population, building a learning community is of great importance to 
ensure all the students' learning, and thus raising a conscious awareness of the use 
of linking talk moves should be necessary. 

The correlation and regression analyses indicate that good accountable talk 
moves had a positive and strong relationship with the level of rigor in the lessons. 
Specifically, the ratings of the academic rigor were highly predicted by the moves 
concerning students' accountability accurate knowledge (AAK) and rigorous 
thinking (ART) among other talk moves. 

One possible explanation of this result may be found in the way that the 
academic rigor was defined in the IQA tool. That is, the IQA tool defined the 
rigorous lesson as one involving a thorough understanding of the text in addition to 
analyzing and interpreting the text. The task of analysis and interpretation of text 
should be accompanied by providing evidence from the text to achieve a high score 
in Academic Rigor on the IQA. Thus, when the discussion was involved in 
providing accurate knowledge, the raters might have considered the lesson as 
rigorous, focusing on a close reading of the text itself. 

In addition, the result that the students' talk moves had more predictability of 
the rigor of the lesson suggests that the students' active participation is a key factor 
to determine the rigor of the lesson. Regardless of the nature of the IQA descriptors, 
the raters appeared to consider a rigorous lesson when they observed the students' 
talk moves rather than the teacher's. 

However, it should be noted that the counterparts of the teacher's moves were 
highly associated with the students' talk moves as shown in the results of the 
correlation analyses. The correlation analyses show that the students' accountability 
to knowledge talk was highly intercorrelated with the teacher's counterpart. This 
was true with accountability to rigorous thinking talk. These results suggest that the 
teacher's talk moves play an important role in generating the interaction with the 
students. In other words, the students' responses often rely on the types of questions 
that the teacher phrased. 
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This finding was congruent with the results from the qualitative analyses of the 
teacher's and students' interaction. As illustrated in the talk move examples, the 
quality of the students' talk was commonly determined by the types of the teacher's 
talk. The qualitative analyses of the teacher and students' interaction revealed that 
the use of the teacher's strategy to keep probing the students' ideas were an effective 
way to encourage students' elaboration. When the teacher failed to reformulate or 
press students to elaborate on their ideas, exchanges tended to be brief and thus did 
not contribute to students' substantial understanding of text. Typically, yes-no 
questions, or one-word answer questions asking surface level facts from the story 
failed to lead students to elaborate their responses. 

The results of this study provide a number of implications for classroom 
interaction for rigorous instruction. First, raising the awareness of explicit use of 
linking talk moves is necessary for building a learning community. A number of 
studies have proved that a discussion is an effective way of socializing intelligence 
for promoting learning (Applebee et al, 2003; Beck et al., 1997; Cazden, 2001; 
Erickson, 1996; Palinscar, 2003). As Palinscar (2003) emphasizes, teachers should 
make explicit the processes of constructing meaning from the text. One way of 
achieving this goal is the explicit use of effective talk moves. The effective use of 
linking talk moves can create a setting that invites more active interaction by 
ensuring that all the participants understand important concepts and procedures 
during the lesson. As Fleyman (1983) emphasizes, common knowledge must be 
reformulated so that the participants can use the information to share in its meaning. 

Second, in order to generate an opportunity for students to elaborate their 
knowledge and logic, teachers need to increase the use of the pressing strategy. As 
Cazden (2001) points out, the traditional IRE structure restrains the teacher and 
students from co-constructing the dialogue due to the nature of the teacher's 
questions. The pressing strategy will provide an opportunity for students to explore 
and reformulate their thinking rather than concluding the interaction. 

Finally, even though it is necessary to be aware of the explicit use of the talk 
moves for accountable talk, there should be a distinction between the formulaic use 
and substantial use. As demonstrated in examples, it was often observed that the 
teacher explicitly used the talk moves (e.g.. How many agree? Do you agree? Why?), 
but she failed to use them effectively. The common case of the formulaic or trivial 
use was found when the teacher answered her own question or gave no wait time to 
the students. 
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Although this study was undertaken to test our hypothesis that the discussion 
types have a significant influence on rigorous reading instruction, some limitations 
are evident. One may be concerned about the effect of the grade level on the quality 
of talk. Although the qualitative analyses presented that younger graders were 
capable of a rigorous discussion using good talk moves, whether some weak talk 
moves may have been due to limitation of the verbal capability of children in the 
younger grades. For a future study, it will be of interest to examine the 
characteristics of talk moves for each grade level. It should also be noted that 
statistical analyses were conducted with a small sample size. Thus, the scope of 
generalization of the results of this study is limited. 

Despite the limitations, this study provided supportive evidence that a 
classroom discourse including listening to others, questioning other's knowledge, 
and exploring one's own thoughts has a positive relationship with the academic 
rigor of reading comprehension. Additionally, if one attempts to assess the quality of 
instruction such as in the IQA toolkit, the quality of classroom talk should be 
considered as a lens to look at the classroom culture. 
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Appendix 



Rubrics of Accountable Talk and Academic Rigor 

Because the current IQA rubrics are still at the developmental stage, the score 
scale of 4 for each rubric is presented for example only. 

Accountable Talk Rubrics 

1. How effectively did the lesson-talk build Accountability to the Learning 
Community ? 



Rubric 2: Teacher's Linking Contributions: Does the teacher support the speaker's 
linking contributions? 



4 


At least 4 points during the discussion, the teacher explicitly connects 
speakers' contributions to each other and shows how ideas /positions 
shared during the discussion relate to each other by revoicing and 
recapping students' ideas. 


Rubric 3: Students' Linking Contributions: Do speakers' contributions link to and 
build on each other? 


4 


At least 4 points during the discussion the students, explicitly connect 
speakers' contributions to each other and show how ideas /positions 
shared during the discussion relate to each other. 


2. How effectivelv did the lesson-talk build Accountabilitv to Knowledge? 

Rubric 4: Asking knowledge: Were contributors pressed to support their 

contributions with evidence? 


4 


There are 4 or more efforts to ask students to provide evidence for their 
contributions, including questions that seemed academically relevant. 


Rubric 5: Providing knowledge: Did contributors support their contributions with 
evidence? (This evidence must be appropriate to the content area — i.e., evidence 
from the text; citing an example, referring to prior classroom experience.) 


4 


At least 4 points, speakers provide accurate and appropriate evidence for 
their claims, including frequent references to the text or prior classroom 
experience. 



3. How effectively did the lesson-talk build Accountability to Rigorous 
Thinking ? 
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Rubric 6: Asking thinking: Were speakers pressed to explain their thinking 
during the lesson? 



4 


There are 4 or more efforts to ask students to explain their reasoning, 
including questions that seemed academically relevant. 


Rubric 7: Providing thinking: Did contributors explain their thinking during the 
lesson? 


4 


There are 4 or more examples of speakers explaining their thinking, using 
reasoning in ways appropriate to the discipline. 


Academic Rigor Rubric (only one rubric is presented) 

Rubric 1: Active Use of Knowledge: Analyzing and Interpreting the text during 
Discussion 


4 


The teacher guides students to engage with the underlying meanings or 
literary characteristics of a text. Students interpret or analyze a text and 
use specific examples from the text and/or cite examples from the text to 
support their ideas or opinions. 
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